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ABSTRACT 

Collaboration among institutions of higher education involves the sharing of financial, 
administrative and infrastructural resources with others through a formal memorandum of 
understanding. There are occasions where due to bureaucratic or political barriers, a formal 
collaborative arrangement may not be possible, however, academic partnerships may foster 
informal collaboration or cooperation among institutions. This paper examines one such informal 
collaboration existing between the University of Botswana and The University of the West Indies 
Distance Education Centre. The authors share the informal collaborative model used to develop a 
quality assurance tool for eLearning and compare approaches in eLearning course development 
and quality assurance procedures at both institutions. 

Keywords: Collaboration; eLearning; course development; quality assurance; developing 
country; blended learning. 


INTRODUCTION 

Michael Moore and Kris Lambert (1996) point out that while distance education has always been 
characterised by communication, new technologies have made collaboration among all 
stakeholders in open and distance learning far easier than before. Definitions of collaboration 
include “an active working partnership supported by some kind of institutional commitment” (Neil 
1981:25). Evidence of formal collaboration in distance education over the years exists in the form 
of associations and consortia such as the Asian Association of Open Universities, the Latin 
American Cooperative Network for the Development of Distance Education and the 
Commonwealth of Learning (COL). While researchers such as Moore and Lambert (1996) and 
Dhanarajan (1998) emphasize the many benefits of formal collaboration, particularly for open and 
distance learning, writers such as Moran (1990) point out that informal and ad hoc agreements 
can also yield benefits for institutions and individuals involved in open and distance learning, 
particularly in developing countries. 

The University of Botswana (UB) and The University of the West Indies (UWI) share similar socio¬ 
economic environments. Similar higher education experiences and existing resources at both 
institutions make formal collaboration an attractive premise. Efforts have been made by the 
administration of UWI to establish a memorandum of understanding with UB, but this has not yet 
materialised. In the meantime, however, academics within both institutions have been using their 
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professional links with colleagues to establish informal working relationships. This paper 
describes one such relationship which emerged out of a joint research project undertaken by 
members of staff of both institutions and which has resulted in the development of quality 
assurance benchmarking criteria for eLearning at both institutions. The paper also examines the 
similarities and differences between the institutions, which reinforce the notion that a more formal 
collaborative arrangement would be beneficial to both institutions. 


BACKGROUND 

The University of Botswana’s experience in use of ICT in education 

The University of Botswana, through the Educational Technology Unit (EduTech) of the Centre 
for Academic Development (CAD), has embarked aggressively on a programme of technological 
transformation over the last three years (Uys, Nleya & Molelu 2003). EduTech has led the 
eLearning (UBel) initiative, where eLearning is defined as the appropriate blend of information 
and communication technologies (ICT) to enhance student-centred, collaborative and lifelong 
learning, combining face-to-face and web-based approaches in teaching and learning. The 
initiative was launched in February 2001, to transform teaching and learning through use of 
eLearning as a blended learning approach. A large part of this technological transformation, 
particularly at start up, has been development of academics in the effective and appropriate use 
of educational technologies. In response to this need the EduTech has developed a programme 
for UB teaching staff consisting of eighteen (18) workshops that lead to an “eLearning Certificate”. 

Substantial progress has been made at UB, with funded research for development of eight 
eLearning pilot courses to establish best practice models and quality assurance measures. In 
addition, the Academic Programme Review Unit (APRU), also in the CAD, has assisted in 
development and testing of standards and criteria for evaluation of these eLearning pilot courses. 
This research and the outcomes of this research to date have been primarily centred on on- 
campus learning because of the level of development that exists within the university and the lack 
of access to ICT for off campus students. 

The Centre for Continuing Education (CCE), designated in 1994 by the Government of Botswana 
as the leading institution for development and delivery of part-time and distance education at the 
tertiary level, has been functioning separately from the on-campus eLearning initiative. 
Programmes offered by the CCE include: Part-time Diploma in Accounting and Business Studies 
through the Extra Mural Unit; Diploma in Primary Education for certificate holders; Diploma in 
Adult Education through the Distance Education Unit; and, non-credit courses in the Public 
Education Unit. Of the three units that comprise the CCE, the only unit that uses distance 
education strategies is the Distance Education Unit (DEU). Programmes offered by DEU are 
primarily a combination of print based, video and audio materials combined with face-to-face 
delivery. Although the basic mode of delivery of courses at the CCE may be similar in some ways 
to the UWIDEC mode of delivery, the level of sophistication in the use of ICT is significantly lower 
in the DEU. 

UWIDEC’s experience in use of ICT in education 

UWIDEC was established in 1996 to expand the university’s initiatives in distance education 
throughout the English-speaking Caribbean. It was thought that the addition of use of a 
combination of technologies could reduce cost, expand the range of programmes and improve 
quality in programmes offered through distance education (Kuboni, Thurab-Nkhosi & Chen 2002). 
Currently, the UWI offers three full undergraduate degree programmes at a distance through its 
thirty (30) centres located in sixteen (16) Caribbean territories. These Centres are administered 
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by the three campus offices located at Mona, Jamaica; Cave Hill, Barbados and St. Augustine, 
Trinidad. UWIDEC does not regard its use of the Web as high level and currently considers 
eLearning as asynchronous learning involving the blending of multimedia technology and web- 
based learning with the more traditional print based delivery and audioconferencing. 

In an attempt to capitalize on use of technologies in educational programmes UWIDEC has 
conducted research on their distance education programmes and the value of the online 
experience. In 2001, staff at the St. Augustine, Trinidad office of UWIDEC embarked on a long¬ 
term research and development project designed to monitor and evaluate the expanded use of 
ICT in programme delivery and to identify and analyse the key factors required to facilitate 
sustained growth of eLearning in the university’s distance education programme. Based on the 
first two phases of this research, it was projected that in order for UWIDEC to successfully 
incorporate web-based learning into its existing mix of course delivery methodologies, it was 
necessary, at least in the initial stages, to focus attention on development in three specific areas. 
The areas were: instructional design/course development, the training and development of course 
writers/coordinators (content experts) in eLearning course development and the training of 
technicians at UWIDEC sites to function in the area of student support (Kuboni, Thurab-Nkhosi 
and Chen 2002). 

While the UWIDEC network provides the connectivity for the online delivery mode from 2000 to 
late 2004 the Web CT learning management system (LMS) was the platform on which the online 
teaching/learning environment was built. In 2004, based on financial considerations, UWIDEC 
took a decision to move to an open source LMS called Moodle for all future online course 
elements. 

With the growing use of web-based technologies in formal education, there is a variety of 
emerging modalities in which the computer-networked environment is being used for instructional 
delivery. UWIDEC’s approach to the use of the Web between 2001 and 2005 can be regarded as 
falling at the lower end of Eastmond’s (1998) continuum of Internet-based distance education. 
Along this continuum, Eastmond identifies three types of Internet use. Type I, at the lower end is 
described as traditional distance learning supplemented with Internet activities. He explains 
further that this type of learning “allows students to participate in e-mail exchanges with 
instructors and other students, supports online research in libraries ... and may also make use of 
online discussion groups...” (p.34). 

Between 2001 and 2005 UWIDEC St. Augustine had a web-based component in eight (8) 
courses, which seemed to fall within Eastmond’s Type 1 for Internet use. These courses were: 

• Introduction to Sociology 

• Caribbean Business Environment 

• Industrial Sociology 

• Topics in Economic Development 

• Operations Planning and Control 

• Current Issues in Educational Administration 

• Principles of Marketing 

• Survey design and analysis 

The online elements of seven of these courses have been available to students since 2001 and 
have been the subject of a research study conducted by UWIDEC St. Augustine. (Kuboni, 
Thurab-Nkhosi and Chen 2001; 2002). 
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Comparison of the UB and the UWIDEC eLearning Approaches 
Similarities 

There are a number of similarities between UB and UWI in regard to eLearning approaches. One 
feature that both institutions have in common is that both institutions, at least until 2004, use 
WebCT as the Learning Management System. This feature is useful as it provides one with 
greater ability to compare approaches and challenges to development and delivery with a view 
towards sharing of strategies that work. As UWIDEC gains experience with the open source 
Moodle system, UB may benefit from collaboration on experiences with this system. 

Both UB and UWI have recognized the need for capacity building of staff members and have 
organized formal training to achieve this. Both institutions have focused on instructional design 
principles as a foundational skill for integration of eLearning into courses and both have 
developed courses to facilitate increased teaching staff skills to utilize a variety of eLearning 
technologies. Because of the need for capacity building of staff, both institutions have integrated 
eLearning elements into pre-existing courses and are conducting research on best practices in 
development, delivery and quality assessment of eLearning courses. Results of the research and 
sharing results of these studies have informed the present systems of course development, 
delivery and quality assessment at both institutions. 

A particular focus of both institutions lies on the instructional design of eLearning courses. In both 
institutions instructional design is seen as a sequential process, which is flexible and practical 
(not linear). The instructional design process for eLearning at UB and UWIDEC takes into 
account the context of both institutions. In this regard, the instructional designers at each 
institution consult with the lecturer interested in eLearning, and together they develop a course 
outline. The lecturer follows the general steps identified in the various models of instructional 
design, and completes an “eLearning Course Concept Development Checklist”, covering all areas 
affected by eLearning, such as content, communication, tutoring, collaboration, assessment and 
quality assurance. 

Both institutions address student support in their staff training but neither institution has 
developed formal programmes for student capacity building to date, although orientation sessions 
for students using WebCT are being conducted at UB at the beginning of each eLearning course 
and orientation sessions are conducted with the distance education students in Trinidad and 
Tobago. Some of the islands, which form part of the UWIDEC network are yet to fully accept 
eLearning as a part of the UWIDEC course delivery strategy and as such are providing some 
resistance to efforts to take the process forward. Resistance by the Distance Education Unit at 
UB is also problematic. Reasons for this resistance need to be explored, however, resource 
limitations may be the primary reason for the lack of initiative in this area. An additional reason for 
resistance may well be pedagogical biases, especially in the case of more traditional thinkers, 
which was demonstrated in the study of online learning at UB where older, higher level 
professionals were more resistant to online learning (Gachago-Giannini, Lee & Thurab-Nkhosi 
2004). 

Differences 

Facilities that promote eLearning are different between UWI and UB in that UWIDEC has 
synchronous (telecommunications centres) and asynchronous technology, including personal 
computers for twenty-five (25) students at all satellite centres (Thurab-Nkhosi 2004) while UB has 
only recently arranged for use of Ministry of Education facilities, including personal computers, in 
some of the satellites throughout Botswana. As a result, use of the Internet is nearly absent in 
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distance education at UB with only those students in the Diploma in Accounting and Business 
having access to the Internet and that access is severely limited. 

Other differences that exist between the approaches to eLearning used by each institution include 
the staff involved in eLearning support. UB relies on an eLearning support team of three people, 
namely an Instructional Designer, a Graphic Designer and a Multi Media Producer for all 650 staff 
members, while UWIDEC relies on three (3) existing curriculum development teams, which were 
initially brought together for developing print-based courses. There is currently only one graphic 
designer, one multi-media producer, and one web developer to be shared among the three teams 
on the campuses at Mona, Jamaica; Cave Hill, Barbados and St. Augustine, Trinidad. 

Furthermore, UB provides workshops, individual consultancy and training for lecturers interested 
in eLearning and the eLearning certificate. While UWIDEC provides training in the principles of 
course design and more recently an orientation to blended learning in the UWIDEC context, the 
main responsibility for training academic staff at UWI rests with a separate department, the 
Instructional Development Unit (IDU), which falls under the Office of the Campus Principal. 
Currently, the main campus is still using the WebCT LMS and as a result the IDU’s training for 
eLearning is centred on WebCT. 

At UB, after the eLearning team completes the eLearning Course Development Checklist, the 
lecturer then works with a multi-disciplinary development team to finalise the eLearning 
activity(ies) chosen. This team includes the lecturer/content expert (academic staff member), 
instructional designer (EduTech), online media developer (EduTech), graphic designer 
(EduTech), and a Library Representative. This is where the process differs at UWIDEC because 
the continued development of the course rests with the content expert and the instructional 
designer and editor, without the assistance of an eLearning development team. Furthermore, at 
UB, support for the development of a well-designed course is given individually and interested 
lecturers/content experts can seek support at any stage of the eLearning course development 
process. This assistance consists of training in development and publishing of online material, the 
development of power-point presentations or any other aspect of integrating ICT in teaching and 
learning (Thurab-Nkhosi, Giannini and Lee 2004). 


COMPARISON OF QUALITY ASSURANCE PROCEDURES IN eLEARNING AT UB AND 
UWIDEC 

Clearly the process used for course development and delivery described above is a very 
important factor in the level of quality of programmes. The similarities and differences and 
lessons learned from these approaches are used by these institutions in development of their 
guidelines for development and delivery, which are discussed in a later section of this paper. In 
this section similarities and differences in quality assurance procedures for eLearning courses are 
discussed. 

Similarities 

Both universities have adopted the definition of quality as “fitness for purpose” (Thurab-Nkhosi 
2004; UB Academic Quality Management Policy 2003), although both UWIDEC and UB may 
review this definition in relation to specific distance education activities. While quality assurance 
systems are in place for on-campus courses both universities have been slow to develop quality 
assurance standards and systems of measurement for distance education courses and 
programmes. Furthermore, these policies, procedures and processes are overseen in both 
institutions by special bodies that have quality assurance as a primary mandate. 
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The global emphasis on a dual approach to quality assurance, with internal and external systems 
as the most common strategies, has been adopted by both institutions (Thurab-Nkhosi 2004; UB 
Academic Quality Management Policy 2003). There are, however, differences in the stage of 
development of quality assurance strategies used for eLearning in the two institutions. 

Differences 

The difference in quality assurance for eLearning courses at the two universities is essentially 
found in the level of the strategies in place. At UWIDEC the strategy is at a development and 
delivery level, while at present at UB the strategy is at the individual course delivery level. In order 
to maximize the lessons learned from each institution both are sharing best practices and lessons 
learned in the less developed area of quality assurance, i.e., individual course delivery or 
programme development level quality assurance is shared by UWIDEC while UB shares the 
lessons learned in their quality assurance in individual course delivery. These activities are 
discussed in the section on the collaborative model that is presently in place. 


WHY COLLABORATION BETWEEN UB AND UWI? 

The following information is provided to support the authors’ notion that there are similarities 
between Botswana and Trinidad and Tobago. It was based on these similar characteristics that 
the authors felt that this could be an opportunity for meaningful collaboration. Trinidad and 
Tobago has a population of 1.3 million, only slightly less than Botswana’s 1.7 million and both are 
considered developing countries. Both Botswana and Trinidad and Tobago are economically 
dependent on non-renewable natural resources, Botswana on diamonds and Trinidad and 
Tobago on petroleum (Mogae 2003). Furthermore, the Caribbean countries that comprise UWI 
are all Anglophone countries as is Botswana. Both countries, until recently, have a 
national/regional university with “satellites”. However, because the West Indies is a region 
composed of many islands, UWI (with over 30 satellites spread throughout sixteen Caribbean 
islands) has greater dispersion of university facilities than does UB. Based in a land locked 
country, UB has only six “satellites”. 

It is likely that these two countries have more contextual elements in common than would two 
countries with vastly different populations or political-economic contexts. Many countries are 
aligned on the basis of these areas of similarities, especially with regard to quality assurance 
(Fourie, Strydom & Stetar 2000). It is because of these significant similarities that collaboration of 
the two countries was considered to be useful in learning about best practices in course 
development and delivery and quality assessment in eLearning. 


THE UB-UWIDEC INFORMAL COLLABORATIVE MODEL 

As stated earlier, the collaborative model used emerged within the context of a research project 
developed by the Academic Programme Review Unit at the University of Botswana to develop 
guidelines for best practices and an instrument(s) for assessment of quality in Web-based 
courses. The research team comprised the staff of the APRU, which included a visiting member 
of UWIDEC staff who was working on a short-term contract with the APRU at the time and a 
member of staff of the eLearning support unit of the Centre for Academic Development. The 
model involved the development of processes based on the research undertaken at UB and the 
application and revision of these processes at UWIDEC, based on the experiences at UB. The 
collaboration therefore progressed through four stages: 

Stage 1: Development and conduct of the UB research project; 

Stage 2: Development and pilot testing of a UB benchmarking tool for quality assurance in 
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eLearning course development; 

Stage 3: Adaptation and pilot testing of a UWIDEC quality assurance tool for eLearning course 
development based on the UB experience; 

Stage 4: Sharing of experiences in using the tools developed and revisions. 

The following section describes these four stages as they relate to a study to develop an 
instrument(s) and processes to measure quality of eLearning courses. 

Stage 1: Development and conduct of the research project 

The purpose of this study was to use benchmarking research to develop indicators of quality in 
eLearning courses at UB and to use these indicators to assess the quality of eight eLearning pilot 
courses developed by faculty members at UB in collaboration with the instructional designer, 
EduTech. Based on the research findings, a best practice model and criteria and processes for 
assessment of future eLearning courses at UB were to be developed. 

The draft instrument to be used for quality assessment was developed by the research team 
based on, but not limited to, benchmarking research done by the following: WebCT Exemplary 
Course Awards Rubric; Billings, Conners & Skiba (2001) research on quality of web-based 
nursing programs; the Educational Technology Unit eLearning Course Design Checklist; and, the 
US based Flashlight Program that conducts research on use of ICT in educational programs. 
Students, designers and other teaching staff of the eLearning pilot courses were asked to 
complete the instrument(s) developed to assess the benchmarks. In addition, ten (10) focus 
group discussions were conducted to obtain qualitative data on the experience of students and 
staff. The qualitative data were used to validate and augment the data obtained through 
administration of the quantitative instrument(s). Furthermore, these qualitative data were 
instrumental in guiding the researchers in adopting the resulting quality assurance processes. 

Quantitative data were analysed using SPSS. Descriptive and inferential statistics were used to 
determine the significance of variables in determining the quality of the eLearning course. 
Qualitative data from focus groups were recorded and transcribed by research assistants. Use of 
a constant comparative method of data analysis was used with categories and themes of 
responses identified. These data were used to further define the significance of variables 
measured in the quantitative instrument(s) in contributing to best practice. Comparison of marks, 
in those courses with a face-to-face counterpart or that have been taught by face-to-face 
previously, were used to determine overall impact of the eLearning mode of delivery to the quality 
of learning and teaching. Finally, all of these data were used in development of guidelines for best 
practice in eLearning at UB. 

Stage 2: Development and pilot testing of a UB benchmarking tool for quality assurance in 
eLearning course development 

The ‘Draft Quality Assessment for Online Learning Rubric’ was initially made up of six distinct 
domains of assessment criteria. The domains emerged from the literature on quality assessment 
of web based programmes and courses, as indicated previously. The number of items in each 
domain is contained in brackets after the domain. The six domains were: needs assessment (2), 
student learning objectives and performance (2), active learning (5), online organization and 
delivery (14), student support (6), and student and course evaluation (11) for a total of 40 items. A 
balance was sought between usability, in terms of convenience and time required to assess the 
quality of the course and validity of the scores, of the completed assessment rubric. For these 
reasons, the researchers attempted to keep the number of items below 50 and developed the tool 
using MS Excel so that scores could be generated automatically and quickly (Appendix A). 
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The levels of performance to be indicated on the rubric were rated: not present (0), weak (1), 
acceptable (2) and exemplary (3). Each of these ratings was given a number as indicated in 
brackets behind the rating. The descriptors for each of the ranks for the criteria were based on 
reports in the literature on performance in web-based and distance learning programmes in both 
developed and developing contexts. The highest possible score for each domain was calculated 
by taking the number of items multiplied by three, i.e., domain one: needs assessment, with two 
items, has 2 X 3 = 6 or a possible maximum score for this domain of six. A maximum total score 
for the combined domains in the rubric was 40 X 3 = 120. 

Pre-testing of the ‘Draft Quality Assessment for Online Learning Rubric’ 

The ‘Draft Quality Assessment for Online Learning Rubric’ for evaluation of eLearning courses 
was tested in April 2003 on four Southern African Development Corporation (SADC) 
environmental courses that were developed through a regional development project (SANTREN) 
by one member of staff of UB staff and the visiting staff member from UWIDEC. The review was 
conducted prior to the courses being implemented. The purpose of this pre-test was to determine 
the usefulness of the rubric and to make modifications as necessary based on the reviewers’ 
suggestions following assessment of the courses. The reviewers that utilized the ‘Draft Quality 
Assessment for Online Learning Rubric’ for evaluation of the SANTREN eLearning courses found 
a number of benefits and weaknesses through this pre-test of the instrument. The most apparent 
strength of the draft rubric was the ease of its use for determining the level of quality of specific 
elements of the courses assessed. Because of the multiple sources of information used in 
determining the criteria to include in the rubric, the authors found they were able to enhance 
clarity of the criteria as well as promote the quantification of performance on each of the criteria. 

Stage 3: Adaptation and pilot testing of a UWIDEC quality assurance tool for eLearning 
course development based on the UB experience 

The curriculum development teams at St. Augustine, Mona and Cave Hill, with permission from 
the team leader of the UB research group, reviewed the UB Benchmarking rubric and arrived at a 
format and specific criteria to be covered in a quality assurance tool for UWIDEC. While the UB 
benchmarking rubric was essentially an evaluation tool to be used at the end of course 
development, it was felt that UWIDEC should develop a quality assurance tool to be used 
throughout the course development and delivery process. The criteria used by UWIDEC were 
informed by international criteria established for measuring quality in distance education, a 
decision that resulted from the differing level of eLearning infused in distance education between 
the two institutions. 

Based on the criteria, and the UB experience with the benchmarking rubric, the UWIDEC quality 
assurance tool was designed to be used during three stages of course development, namely pre- 
production, production and evaluation. A series of teleconferences were held to arrive at the draft 
quality assurance tool (see Appendix B). It was decided that this tool would be used on a pilot 
basis as new online courses were developed so that revisions to the tool could be done based on 
practical experience. To date the tool has been used by the St. Augustine and Mona campuses. 

Stage 4: Sharing of experiences in using the tools developed and revisions 

Based on experiences of the two institutions, the authors believe that lessons learned through 
research and practice at each institution could continue to be shared to develop more useful 
guidelines for eLearning programme development, delivery and methods to measure quality of 
the programmes. It is felt that sharing research findings from UWIDEC’s long-standing distance 
education experience and UB’s more developed on-campus use of eLearning strategies will 
benefit both institutions in their quest for further development of eLearning in these two 
developing country settings. 
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At UB, what has been learned from the eLearning pilots has been developed into Draft Guidelines 
for Development and Delivery of eLearning Courses (Appendix C). In addition, further 
development of the benchmarking rubric to include data from a revised interview schedule for 
focus group discussions (student and course expert) and revised student online questionnaire is 
being implemented and tested. 

At UWIDEC the following has been learnt from application of the quality assurance tool. 

Adequate Time Frame 

A course must be developed within a reasonable time frame so that the tool can be applied early 
enough to allow for changes in the course development process. In other words, courses that are 
being developed as they are delivered do not allow for application of the tool and this is clearly 
not best practice. To benefit from the outcomes of the use of the tool, there must be adequate 
time for planning the course to allow for changes, questioning during the development and 
delivery process, reflection on the process and finally amendments when required. 

Consistent application 

To be effective the quality assurance tool must be applied consistently throughout the process. 
Application during the planning and development stage, while omitting the evaluation stage for 
example, devalues the tool and reduces its effectiveness. It should also be recognized that a 
decision to use the tool requires application to ALL courses with an online component and not just 
to the ones that UWIDEC may have more time to work with. This requires commitment by staff on 
all campuses involved in the course development and delivery process. 

Responsibility for applying/using the tool 

Specific individuals should be assigned responsibilities for applying the tool in order to ensure 
consistency of delivery. The various levels or phases of the planning and delivery process to 
which the tool can be applied generally determine the person(s) responsible for application. For 
example, during the course planning stage, the curriculum specialist should assume responsibility 
for ensuring the criteria identified for quality are met. Clear guidelines on who is responsible for 
what activity must be given at each phase of course development. 

Follow-up 

Decisions and action must be taken on data obtained from the quality assurance tool, and clear 
time frames developed and specific actions taken to address any problems or challenges 
identified. Online environments can allow for changes and adjustments to courses with relative 
ease. Advantage must be taken of this characteristic. 


CONCLUSION 

In this paper the authors have compared approaches to eLearning course development and 
quality assurance at the University of Botswana (UB) and The University of the West Indies 
Distance Education Centre (UWIDEC). The paper further describes a collaborative model where 
lessons learned in these two institutions are shared and compared with the result being 
guidelines for course development and delivery and quality assessment of eLearning courses at 
UB and UWIDEC. UB and UWIDEC have had similar learning experiences and both institutions 
will be sharing strategies to strengthen the quality assurance and course development processes 
based on lessons learned. 
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APPENDIX A 

UB QUALITY ASSESSMENT FOR ONLINE LEARNING RUBRIC 

Evaluation Phase 1 (after production) 


BENCHMARKING RUBRIC 

# 

Benchmark 

0 

Weak Performance 

1 

Acceptable 

Performance 

2 

Exemplary 

Performance 

3 

Comments 

Needs Assessment Criteria 

1 

Needs assessment 


There is little evidence 
in the course materials 
that student and 
society needs have 
been considered in 
the development of 
the course 


The content seems to 
address society and 
learner needs but there 
is no indication that a 
formal needs 
assessment was 
conducted 


Clear rationale for 
selection of content 
and its relation to 
society and learner 
needs is 

communicated in the 

course 



~~2 

Target group 


The target group for 
this course is not 
clearly communicated 
in the course 


The target group is 
clearly stated in the 
course materials 


Rationale for selection 
of this target group is 
communicated in the 
course materials 





0 


0 


0 


0 


Needs Assessment Criteria Total: 

0 


Student Learning Objectives and Performance Criteria 

3 

Clear student 
learning objectives 


Student learning 
objectives stated in 
vague terms 


Student learning 
objectives stated clearly 


The student has 
inputs into the 
objectives, i.e., 
student objectives 



~4 

Clear performance 
criteria 


Unclear pefomance 
criteria 


Clear performance 
critera 


Clear, manageble 
performance criteria 
with conditions for 
performance, i.e., 
rubrics 





0 


0 


0 


0 



Student Learning Objectives and Performance Criteria Total: 

0 


Active Learning Criteria 

5 

Active learning, i.e., 
case study, problem 
based, anchored 
learning 


Only one example of 
active learning for the 
course 


On average one 
example of active 
learning per unit or 
module of the course 


The entire course is 
based on students 
actively interacting 
with the content 



”6 

Opportunity for 
student to student 
interaction 


Limited 

encouragement for 
student to student 
interaction 


At least weekly 
interaction is expected 
between students 


Nearly continuous 
interaction is expected 



~7 

Opportunity for 
student to instructor 
interaction 


Limited expectations 
for student to 
instructor interaction 


At least weekly 
interaction is expected 
between student and 
instructor 


Nearly continuous 
interaction is expected 
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8 

Further resources, 
e.g. links, 

presentations, audio, 
video files 


Further resources 
used in a limited way 


Opportunities for 
students to access 
further resources 


Further resources 
engage students in 
the learning process 



~9 

Appropriate use of 
technology tools for 
the objectives 


Tools not clearly 
related to the learning 
objectives 


Tools relate to the 
learning objectives 


Tools enhance 
learning of the content 
and meeting 
objectives 





0 


0 


0 


0 



Active Learning Criteria Total: 

0 


Online Organization and Delivery Criteria 

10 

Homepage 
(welcome, 
information & 
uidance) 


The homepage is 
available but provides 
little welcome 
information nor 
guidance 


The homepage provides 
some welcome, 
information and 
guidance 


The homepage not 
only provides 
information and 
guidance, but it is 
engaging 



TT 

Course outline 


Limited information in 
the course outline 


Most necessary 
information is in the 
course outline 


Provides all 
information required of 
the student in the 

course 



12 

Course schedule 


Limited information in 
the course schedule 


Most information is 
available in the course 
schedule 


All information is 
avialable on the 
course schedule, e.g, 
assignment and 
assessment dates 



13 

Information provided 
to guide student 
through the learning 
process (Help) 


Limited material to 
guide the student’s 
learning and difficult to 
access 


Some material is 
provided that guides 
student learning 


Consistent guidance 
available and easy to 
access 



14 

Ease of navigation 
through course 
components 


Difficulty navigating 
through course 
components 


Some difficulty 
navigating through 
course components 


Easy to navigate 
through the course 
components 



15 

Navigation through 
content 


Difficulty navigating 
through the course 


Some difficulty 
navigating through the 
content 


Easy to navigate 
through the content 



16 

Content in 

manageable 

segments 


Easily become 
overwhelmed by the 
amount of content 


Most segments do not 
overwhelm the student 


All segments have 
manageable amounts 
of information 



17 

Mechanisms used for 
active learning are 
cohesive with 
content 


Little relationship 
between the content 
and the learning 
activities 


Content and learning 
activities are cohesive 


Learning activities are 
part of the course 
content delivery 



18 

Aesthetic design of 
course 


Course design 
rudimentary 


Course design aesthetic 


Course design very 
appealing visually 



19 

Accessibility issues 
are addressed 


Few alternative 
access possiblilities 


Some alternative access 
possiblilities 


All course materials 
are accessible for 
people facing 
connection problems 



20 

Technology tools 
used are appropriate 
for the content and 
objectives 


Tools sometimes 
inappropriate for 
content and objectives 


Tools appropriate for 
content and objectives 


Tools enhance 
learning of the content 
and ability to meet 
objectives 
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21 

Components of the 
course (objectives, 
instructional 
strategies and 
assessment 
techniques) are 
closely aligned 


Little alignment 
between course 
components 


Most course components 
are aligned 


All course 
components are 
clearly aligned 



22 

Consistency in the 
layout of course 
pages 


Little consistency in 
the page layout 


Most page layout 
consistent 


All page layouts 
consistent 



23 

Overall consistencies 
and cohesiveness in 
the course design 


Many inconsistencies 
in course design 


Few inconsistencies in 
course design 


Course design is 
consistent throughout 





0 


0 


0 


0 



Online Organization and Delivery Criteria Total 

0 


Student Support Criteria 

24 

Course orientation 
provided 


Orientation not well 
organized 


Organized orientation 
provided 


Well organized 
orientation 



25 

Adequate tutors for 
the course* 


More or better trained 
tutors would enhance 
learning 


Adequate number of 
trained tutors 


Adequate, well trained 
tutors 



26 

Communication tools 
used in the course 


Inadequate or 
inappropriate 
communication tools 
used in the course 


Adequate or appropriate 
communication tools 
used 


Adequate and 
appropriate 
communications tools 
used 



27 

Consideration of 
different learning 
styles 


Little consideration is 
given for different 
learning styles 


Some consideration is 
given for different 
learning styles 


Learning styles are 
considered throughout 
the material 



28 

Opportunities for 
remediation 


Few opportunties for 
student remediation 


Many opportunities for 
remediation 


Most assignments/ 
assessments provide 
opportunities for 
remediation 



29 

Opportunities for 
student advisement 


Few opportunties for 
student advisement 


Some opportunties for 
student advisement 


Many opportunties for 
student advisement 





0 


0 


0 


0 



Student Support Criteria Total 

0 


Student and Course Assessment and Evaluation Criteria 

30 

Assignments 
congruent with 
course objectives 


Not all assignments 
are clearly related to 
objectives 


All assignments are 
related to objectives 


Objectives are 
assessed through a 
minimum number of 
assignments 



3? 

Assignments clearly 
communicated 


Not clearly 
communicated 


Clearly communicated in 
course materials 


Clearly communicated 
and explained in 
course materials 



32 

Assignments/ 
assessments 
appropriate for target 
group 


Some assignments 
not appropriate for 
target group 


Most assignments are 
appropriate for the target 
group 


All assignments are 
appropriate for target 
group 



33 

Assignments 
encourage critical 
thinking 


Little critical thinking 
required 


Some critical thinking 
required 


Assignments cannot 
be done without 
critical thought 
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34 

Assignments 
encourage problem 
solving 


Little problem solving 
required 


Most assignments are 
based on a problem 
scenario 


All assignements are 
based on a problem 
scenario 



35 

Performance criteria 
for assignments/ 
assessments shared 
with students 


Few of the criteria are 
shared 


Many of the criteria are 
shared 


All of the criteria are 
shared 



36 

Opportunities for self 
assessment 


Few opportunities for 
self assessment 


Many opportunities for 
self-assessment 


All assignments 
include an opportunity 
to perform self- 
assessment 



37 

Opportunities for 
peer (student) 
assessment 


Few opportunities for 
peer assessment 


Many opportunities for 
peer assessment 


All assignments 
include an opportunity 
to perform peer 
assessment 



38 

Opportunities for 
student input into 
assessment criteria 


Few opportunties for 
student input into 
assessment criteria 


Many opportunties for 
student input into 
assessment criteria 


Student input into all 
assessment criteria 



39 

Multiple assessment 
strategies used 


Two or less methods 
of assessment used 


Between 3 and 4 
assessment methods 
used 


More than five 
methods of 
assessment used 



40 

Course evaluation 


Inadequate or 
inappropriate tool for 
couse evaluation 


Adequate and 
appropriate course 
evaluation tool 


Wide variety of 
evaluation tools used 





0 


0 


0 


0 



Student and Course Assessment and Evaluation Criteria Total 

0 
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APPENDIX B 

UWIDEC QUALITY ASSURANCE TOOL 



Activity 

Person(s) 

Responsible 

Time 

Frame 

Assumptions 

Measurable 

output 

Remarks 

Score 


Phase 1: Course Concept (planning) 

1 

An eLearning course 
development checklist has 
been completed 

Course 
developer and 
instructional 
designer 

will be set 
when course 
writer's 
contract is 
being signed 

a separate, 
comprehensive 
tool that 
identifies the 
elements in a 
course plan 
exists. 

a course 
concept 
outlining choice 
& combination 
of media, 
content areas, 
assessment etc 



2 

Clear student learning 
objectives have been 
developed 

Course 
developer and 
instructional 
designer 



Student 
learning 
objectives 
stated clearly in 
the course plan 



3 

Clear performance criteria set 

Course 
developer and 
instructional 
designer 



Assignments, 
rubrics for 
students 



4 

The course plan includes 

Active learning, i.e., case 
study, problem based, 
anchored learning 

Course 
developer and 
instructional 
designer 






5 

The plan includes opportunity 
for student to student 
interaction 

Course 
developer and 
instructional 
designer 






6 

The plan includes opportunity 
for student to instructor 
interaction 

Course 
developer and 
instructional 
designer 






7 

The plan includes the use of 
further resources, e.g. links, 
presentations, audio, video 
files 

Course 
developer and 
instructional 
designer 






8 

The plan includes the 
appropriate use of technology 
tools for the objectives 

Course 
developer and 
instructional 
designer 







Phase 2. Production Preparation 

9 

Draft course content has been 
prepared in manageable 
segments based on the course 
concept 

course developer 
and ID 






10 

Draft content and course 
concept have been reviewed 

Instructional 

designer 






11 

draft content has been edited 

editor 






12 

relevant copyright information 
has been requested 

editor 






13 

A consistent, visually 
appealing course design has 
been developed 

graphic designer 
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14 

Website structure has been 
defined, i.e. areas for course , 
pages in these areas 

web designer 






15 

HTML pages have been 
developed and uploaded 

web designer 






16 

The homepage not only 
provides information and 
guidance, but it is engaging 

web designer 
and graphic 
designer 






17 

The course outline Provides all 
information required of the 
student in the course 

ID and CD 






18 

Course schedule contains all 
information on assignments 
and assessment dates 

ID and CD 






19 

Consistent guidance available 
for student and easy to access 

web designer 






20 

Easy to navigate through 
course components 

web designer 






21 

Easy to navigate through the 
content 

ID 






22 

All segments have 
manageable amounts of 
information 

ID and CD 






23 

Learning activities are part of 
the course content delivery 

ID and CD 






24 

A consistent, visually 
appealing course design has 
been developed 

Graphic 

Designer 






25 

Accessibility issues are 
addressed 

Web designer 







Phase 3: Student Support Criteria 

26 

Course orientation designed 

team 






27 

Tutors assigned for the course 







28 

Communication tools 
developed for the course 

team 






29 

Response time for 
communicationdecided on 

team 






30 

Learning styles are considered 
throughout the material 

Id and CD 






31 

All chats and discussions will 
be moderated 

CD 







Student and Course Assessment and Evaluation Criteria 

32 

Assignments developed to 
encourage critical thinking 

CD and ID 






33 

Self assessment activities 
have been developed 

CD and ID 






34 

Opportunities for student input 
into assessment criteria 

CD and ID 






35 

Course evaluation 

team 
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APPENDIX C 

GUIDELINES FOR DEVELOPMENT AND DELIVERY OF ELEARNING COURSES AT UB 

The following guidelines should be consulted and used in the development of eLearning courses 
in order to ensure the highest quality of the course. 

I. The course design and planning process 

a. Plan course carefully, consult with EduTech, use course outline template 

b. Before starting to plan your course, research existing courses on the WWW or through 
personal contacts 

c. Experiment, you need to gather your own experiences, find your own way of using 
eLearning but also consult departmental/divisional/unit strategies, where existing 

d. Use a step by step approach to familiarise yourself with possibilities of eLearning, start 
with a few elements of eLearning and progressively add new elements every semester 

e. Consider using eLearning, preferably after your students have taken general computer 
literacy courses, e.g., GEC121 or 122. 

f. Analyse your content and identify gaps that eLearning could fill in, e.g., simulations, 
visualisations, graphics - research on the Web, where there are large quantities of such 
materials available 

g. eLearning can facilitate international cooperation, e.g., international colleagues can serve 
as experts or can link their students with your students to discuss common topics of 
interest using online discussion forums 

h. Contact the professional world for experience exchange and professional input 

i. If you plan collaboration activities, make sure students have enough time for it 

j. If you want students to research on the Internet, provide “pathfinders”, web sites as 
starting points, to make their research easier 

k. If you want students to research on the Internet, make sure you give them guidelines on 
how to evaluate a website (e.g., http://www.library.cornell.edu/olinuris/ref/webcrit.html) 

l. Be aware of level of computer skills of your students, if they have little or none, they will 
need a high level of personal support from you 

m. Be aware that one of your course objectives could be increased computer and/or 
information skills and integrate this in your assessment scheme, therefore, include 
eLearning in your assessment strategy from the very beginning, not just as an additional 
resource (e.g., allocate marks for continuous assessment or life-long learning skills 
developed through eLearning) 

n. Just provide what is of real added value to your students, the majority won’t access 
anything else. 

o. Think of anchors/metaphors to personalise your content, like illustrations, stories, 
scenarios or case studies, it enhances identification with course content and makes the 
content livelier. The graphic designer will support you in the development process. 

p. Think about how you will allocate time for lecture hours and lab 
sessions/tutorials/practicals. You might want to reduce lecture hours and increase lab 
hours (if this is allowed) if you provide eLearning opportunities in your lab 
sessions/tutorials/practicals. Consider that one lecturer hour is the equivalent of a 2-3 
hour lab session/tutorial/practical. 

q. Be aware that using eLearning will require more time than face-to-face teaching. Just 
logging in will require some time at the start of sessions. 

r. Be creative and playful, eLearning provides many opportunities to enhance your teaching 
and to make it more interactive 

s. Relate content to contemporary, relevant issues 
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t. Be aware of intellectual property and copyright issues, especially if you are providing 
offline web content 

u. Set specific timeframes for assignments and assessments and make sure these are 
reflected in the online environment 

II. The course development process 

a. A group development process for eLearning is most effective. Various multi-disciplinary 
roles are necessary within an elearning development team. Team members, however, 
should have the freedom to support and build each other up according to their strengths. 

b. Possible roles can be: 

• The role of the sponsor is necessary for the provision of necessary funding, wider 
support and overall project control. This will be, in most cases, your Head of 
Department. 

• You as the subject matter expert (SME) are integral in the development process by 
providing content but also in developing materials electronically. The eLearning 
support team takes over different tasks: instructional design, graphic design, and 
multi-media development. 

• Ideally you should organise a peer review by colleagues, otherwise try to get 
feedback by colleagues, students, the Academic Programme Review Unit, EduTec, 
etc. 

c. Use the official UB Learning Management System for your online course, e.g., currently 
UB employs WebCT. 

d. Select media to be developed based on your skills but also on the skills you would like to 
acquire 

e. Keep ease of updates in mind: online content/html pages are easier to update than, for 
instance, videos 

f. Use benefits of online content, e.g., pictures, colours, animations 

g. If you develop online content make sure you follow online editing guidelines, e.g., 

• Divide content into small “learning nuggets”, don’t put too much content on one page 

• Put the most important information on top (content bubbles to the top) 

• Don’t use long sentences, use a conversational style, avoid technical terms and 
language 

• Use colours & graphics 

• Use structured content, like bullets 

• Use headings and put keywords in bold font 

• Embed questions, assignments, case studies, examples 

• Define a common structure for your modules 

h. If you have decided on a specific sequence for your online material, keep to it. Students 
get used to it and know what to expect (e.g., introduction, contents, links, self-tests) 

i. If you provide downloadable content make sure it is printer-friendly and your students 
know how to download it and have the resources to print documents 

j. Start developing content in advance, it might take time at the beginning to develop online 
content, e.g., use the long break before new academic year starts 

k. Consider providing the content also offline, e.g., download websites, save content on a 
floppy disk or on paper in case the network fails you 

l. Be aware of file sizes, the network might be very slow at certain times 
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III. The Course delivery and management (including online communications) 

a. Use a blended learning approach so that you do not depend entirely on electronic media 
and so that each technology is being used for its strengths 

b. Have contingency plans in case the network fails you 

c. Think of how your students will access your eLearning course, e.g., in the SMART 
classroom (book in advance) or in their independent learning time. If you have to rely on 
students finding access independently consider that you might not be able to rely on 
them getting access 

d. Be specific at the start of the course about expectations, objectives, assessment 
strategies and the eLearning aspects of the course 

e. Book EduTech staff for one or more orientation classes for your students or teaching 
assistants in the lab that they will be using 

f. Consider a progressive use of tools and give deliberate guidance 

g. Ideally, meet with students regularly in a laboratory 

h. At the beginning of the course make sure that all your students are able to log on 

i. Decide carefully whether to use the WebCT e-mail tool or your UB mopipi e-mail address. 
Using both can lead to confusion. We suggest you use WebCT, but make sure you 
regularly check your inbox and reply to students’ e-mails in time (feedback time should 
not exceed 48 hours) 

j. If you use the online discussion forum you need to make it an explicit part of the course 
expectations, including assessment, where appropriate, giving clear guidelines on what 
students should do (e.g., length and frequency of postings, topic definition, how to do a 
summary effectively, etc.) 

k. Consider using a paced rhythm of delivery, upload content week by week 

l. Update content regularly, specifically check if links are still available 

m. If you use online quizzes provide your students with a possibility to try it out first 

n. Mix different kind of question sets, e.g., multiple choice and open-ended questions based 
on your content and objectives. 

o. Provide a questions bank for random generation of question sets if you want to offer 
multiple attempts to submit quizzes. 

p. You can use eLearning for managing large classes, e.g., submission of assignments and 
for ensuring standardised content, but you need to plan it very carefully and involve 
teaching assistants 

q. Don’t raise expectations you are not absolutely sure one can guarantee 

r. Pair students, where possible, that are more computer savvy with students with less 
computer skills 

s. Constantly refer/remind/motivate students to access online content; integrate online 
content in your face-to-face classes. 

IV. Support required: EduTech, Library, Information Technology (IT), Head of Department 
(HOD) and other Faculty members 

a. EduTech provides the following support: 

• Training through the CAD eLearning Certificate 

• Instructional design 

• Media development 

• Graphic design 

• Induction classes for online students 

• Teaching and learning equipment where possible 

• Bookings in the SMART Classroom 




Collaboration for eLearning in developing countries 127 


b. Approach EduTech in time for any support that you need 

c. Approach other faculty members who have already utilized eLearning strategies 

d. Make your HOD aware of your work in eLearning and get support from your HOD 

e. The library can help you in finding resources in the library and on the WWW through your 
subject librarian 

f. IT: Know in advance who is responsible for the laboratories you are using and inform this 
person in advance of your needs. Check in advance whether technology is working in 
your laboratories. 

V. Infrastructure and equipment 

a. Use lab sessions to provide adequate computer access 

b. Ideally use department/faculty labs 

c. Schedule lab sessions when computer labs are likely to be free 

VI. Evaluation / Quality Assurance 

a. Use standardised online student questionnaire at end of the course, you can add 
questions if you want 

b. Consider a focus group discussion with selected students 

c. Consult the Academic Programme Review Unit to perform an eLearning course quality 
review using the Quality Assessment for Online Learning Rubric for feedback on overall 
quality of the course 


Copyright for articles published in this journal is retained by the authors, with first publication rights granted 
to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper 
attribution, in educational and other non-commercial settings. 


Original article at: http://ijedict.dec.uwi.edu//viewarticle.php?id=31&layout=html 



